Metasurface refers to a nano-optical device that achieves unprecedented properties of light using a structure much smaller than the wavelength of light. Nano-optical devices control the characteristics of light at the micro level, and can be used for LiDAR beam steering devices used for autonomous driving, ultra-high-resolution imaging technology, optical properties control of light emitting devices used in displays, and hologram generation. . Recently, as the expected performance of a nano-optical device increases, interest in optimizing a device having a free structure in order to achieve a performance far exceeding that of the device structure in the past is increasing. This is the first case of solving a problem with a large design space such as a free structure by applying reinforcement learning.